Papers with attention Key-Value (KV) cache
Quantize What Counts: More for Keys, Less for Values (2026.findings-acl)
Copied to clipboard
| Challenge: | Empirical evaluations across various prominent LLMs and benchmarks show that key-favored allocations retain up to 98.3% accuracy compared to uniform allocations (e.g., 4-bit keys, 2-bit values). |
| Approach: | They propose two theorems that anchor mixed-precision KV quantization in the intrinsic geometry of Transformer models. |
| Outcome: | Empirical evaluations show that key-favored allocations retain up to 98.3% accuracy while conserving memory. |